AFRL-ML-WP-TP-2006-495 

CASE  STUDY  FOR  NEW  FEATURE 
EXTRACTION  ALGORITHMS, 
AUTOMATED  DATA 
CLASSIFICATION,  AND  MODEL- 
ASSISTED  PROBABILITY  OF 
DETECTION  EVALUATION 
(PREPRINT) 


J.C.  Aldrin  and  J.S.  Knopp 


SEPTEMBER  2006 


Approved  for  public  release;  distribution  is  unlimited. 


STINFO  COPY 

The  U.S.  Government  is  joint  author  of  this  work  and  has  the  right  to  use,  modify, 
reproduce,  release,  perform,  display,  or  disclose  the  work. 


MATERIALS  AND  MANUFACTURING  DIRECTORATE 
AIR  FORCE  RESEARCH  LABORATORY 
AIR  FORCE  MATERIEL  COMMAND 
WRIGHT-PATTERSON  AIR  FORCE  BASE,  OH  45433-7750 


The  public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  searching  existing  data 
sources,  gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of 
information,  including  suggestions  for  reducing  this  burden,  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188),  1215  Jefferson 
Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  any  penalty  for  failing  to  comply  with  a 
collection  of  information  if  it  does  not  display  a  currently  valid  OMB  control  number.  PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 


1.  REPORT  DATE  (DD-MM-YY) 

2.  REPORT  TYPE 

3.  DATES  COVERED  (From  -  To) 

September  2006 

Conference  Paper  Preprint 

4.  TITLE  AND  SUBTITLE 

CASE  STUDY  FOR  NEW  FEATURE  EXTRACTION  ALGORITHMS, 
AUTOMATED  DATA  CLASSIFICATION,  AND  MODEL-ASSISTED 
PROBABILITY  OF  DETECTION  EVALUATION  (PREPRINT) 

6.  AUTHOR(S) 

J.C.  Aldrin  (Computational  Tools,  Inc.) 

J.S.  Knopp  (AFRL/MLLP) 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Computational  Tools,  Inc.  Nondestructive  Evaluation  Branch  (AFRL/MLLP) 

4275  Chatham  Avenue  Metals,  Ceramics  and  NDE  Division 
Gurnee,  IL  6003 1  Materials  and  Manufacturing  Directorate 

Air  Force  Research  Laboratory,  Air  Force  Materiel  Command 

_ Wright-Patterson  AFB,  OH  45433-7750 _ 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

Materials  and  Manufacturing  Directorate 
Air  Force  Research  Laboratory 
Air  Force  Materiel  Command 
Wright-Patterson  AFB,  OH  45433-7750 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  is  unlimited. 

13.  SUPPLEMENTARY  NOTES 

Conference  paper  submitted  to  the  Proceedings  of  33rd  Annual  Review  of  Progress  in  Quantitative  Nondestructive 
Evaluation  (QNDE)  2006 

The  U.S.  Government  is  joint  author  of  this  work  and  has  the  right  to  use,  modify,  reproduce,  release,  perform,  display, 
or  disclose  the  work. 

Paper  contains  color.  PAO  Case  Number:  AFRL/WS  06-2406;  Date  cleared:  1 1  Oct  2006. 

14.  ABSTRACT 

This  paper  explores  feature  extraction  algorithms  for  crack  characterization  in  eddy  current  inspection  of  fastener  sites. 
A  novel  feature  extraction  method  fitting  approximate  models  to  data  associated  with  geometric  part  features  addressing 
adjacent  fastener  sites  and  panel  edges  are  developed.  Data  classification  methods  in  the  circumferential  direction 
around  fastener  sites  are  developed  to  better  characterize  fatigue  cracks  with  improved  noise  invariance.  Model-assisted 
probability  of  detection  results  are  presented  highlighting  the  benefit  of  automation  in  NDE. 


15.  SUBJECT  TERMS 

eddy  current,  fastener  sites,  feature  extraction,  models,  probability  of  detection  (POD) 


Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std.  Z39-18 


18.  NUMBER 
OF  PAGES 

14 


19a.  NAME  OF  RESPONSIBLE  PERSON  (Monitor) 
Jeremy  Knopp 

19b.  TELEPHONE  NUMBER  (Include  Area  Code) 
N/A 


16.  SECURITY  CLASSIFICATION  OF: 


a.  REPORT 


b.  ABSTRACT 


Unclassified  Unclassified 


c.  THIS  PAGE 

Unclassified 


17.  LIMITATION 
OF  ABSTRACT: 

SAR 


5a.  CONTRACT  NUMBER 

F33615-03-D-5204 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

621Q2F _ 

5d.  PROJECT  NUMBER 

4349 _ 

5e.  TASK  NUMBER 

41 _ 

5f.  WORK  UNIT  NUMBER 

05 _ 

8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 


10.  SPONSORING/MONITORING 
AGENCY  ACRONYM(S) 

AFRL-ML-WP 

11.  SPONSORING/MONITORING 
AGENCY  REPORT  NUMBER(S) 

AFRL-ML-WP-TP-2006-495 


l 


CASE  STUDY  FOR  NEW  FEATURE  EXTRACTION  ALGORITHMS, 
AUTOMATED  DATA  CLASSIFICATION,  AND  MODEL-ASSISTED 
PROBABILITY  OF  DETECTION  EVALUATION 


John  C.  Aldrin1  and  Jeremy  S.  Knopp2 
'Computational  Tools,  Gurnee,  IL  60031,  USA 
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Abstract.  This  paper  explores  feature  extraction  algorithms  for  crack  characterization  in  eddy  current 
inspection  of  fastener  sites.  A  novel  feature  extraction  method  fitting  approximate  models  to  data 
associated  with  geometric  part  features  addressing  adjacent  fastener  sites  and  panel  edges  are  developed. 
Data  classification  methods  in  the  circumferential  direction  around  fastener  sites  are  developed  to  better 
characterize  fatigue  cracks  with  improved  noise  invariance.  Model-assisted  probability  of  detection 
results  are  presented  highlighting  the  benefit  of  automation  in  NDE. 

Keywords:  eddy  current,  fastener  sites,  feature  extraction,  models,  probability  of  detection  (POD) 

PACS:  81.70.Ex 

INTRODUCTION 

The  characterization  of  cracks  around  fastener  sites  in  multi-layer  structures  continues 
to  be  an  important  problem  for  the  NDE  community.  To  improve  eddy  current  techniques 
for  this  problem,  computational  methods  encompassing  numerical  models,  novel  data 
analysis  and  classification  algorithms,  and  design  optimization  tools  are  proposed  with  new 
probe  designs.  Recently,  the  development  and  validation  of  numerical  models  based  on  the 
volume  integral  method  [1]  for  the  eddy  current  fastener  crack  problem  have  been 
presented  [2],  This  design  strategy  is  directed  at  improving  the  source  field  characteristics, 
measurement  sensor  design,  and  data  analysis  and  classification  algorithms. 

Due  to  the  complexity  of  aircraft  structures,  a  variety  of  coherent  noise  signals  can 
hinder  characterization  of  cracks  around  fastener  sites.  For  example,  although  an 
asymmetric  response  observed  for  a  fastener  site  in  an  eddy  current  image  from  2D  raster 
scan  data  is  typically  used  to  distinguish  crack  and  no  crack  conditions,  there  are  several 
potential  sources  of  coherent  noise  features  with  similar  asymmetric  responses.  In 
particular,  irregularities  in  the  fastener  hole  fit  condition  such  as  asymmetric  gaps  between 
the  fastener  and  hole  due  to  shifted  or  skewed  fasteners,  or  oblong  holes  due  to  poor 
drilling  can  produce  asymmetric  image  results.  In  addition,  variability  in  probe  lift-off  due 
to  the  scanning  system  hardware  alignment  or  part  surface  conditions  and  the  irregular 
quality  of  windings  in  eddy  current  probe  can  also  be  sources  for  measurement  asymmetry. 
Lastly,  the  presence  of  adjacent  fastener  sites,  part  edges  and  subsurface  structures  can  also 
produce  localized  features  around  the  fastener  site  and  complicate  the  data  interpretation 
problem.  The  presence  of  these  examples  of  coherent  noise  can  increase  false  call  rates 
and  likewise  limit  crack  sizing.  Thus,  there  is  a  need  to  develop  advanced  data  analysis 
approaches  and  reliable  features  that  are  sensitive  to  the  crack  condition  yet  invariant  to 
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such  coherent  noise  signals  also  present  in  real  data.  The  use  of  invariant  features  has  been 
shown  to  be  valuable  for  other  problems  in  eddy  current  nondestructive  evaluation  [3],  In 
addition,  statistical  approaches  to  estimation  of  image  features  have  been  proposed  and 
investigated  for  NDE  applications  [4],  The  primary  objective  of  this  work  is  to  develop 
model-based  feature  extraction  algorithms  and  automated  data  analysis  routines  for  crack 
characterization  with  invariance  to  noise  features  for  eddy  current  inspection  of  fastener 
sites.  Two  separate  feature  extraction  methods  are  presented:  1)  to  address  features 
associated  with  fatigue  cracks  in  the  presence  of  asymmetric  noise  at  the  fastener  site  and 
2)  to  address  fitting  approximate  models  to  data  associated  with  geometric  part  features 
including  adjacent  fastener  sites  and  panel  edges.  Demonstrations  of  both  methods  are 
presented  with  good  results  for  simulated  and  experiment  data.  Promising  features 
indicating  the  potential  for  sizing  subsurface  cracks  are  also  presented. 

To  validate  the  performance  of  new  inspection  techniques  incorporating  automated  data 
analysis  algorithms,  probability  of  detection  (POD)  studies  are  performed  including 
preparation  of  multiple  samples  that  closely  represent  the  geometry  of  the  structure.  These 
samples  include  a  statistically  significant  number  of  inspection  opportunities  containing  a 
distribution  of  flaw  sizes.  The  preparation  of  POD  samples  and  the  process  to  acquire  data 
from  these  samples  is  often  very  time  consuming  and  expensive;  thus,  model-assisted  POD 
(MAPOD)  evaluation  methodologies  have  recently  been  proposed  [5].  Through  the  use  of 
computer  simulations  to  model  the  inspection  process  and  statistical  methods  to  determine 
the  POD  from  a  combination  of  experimental  and  simulated  data,  significant  cost  and  time 
savings  can  be  achieved.  However,  additional  work  concerning  the  validation  of  models 
and  the  MAPOD  methodologies  is  needed  to  ensure  the  reliability  of  this  approach.  In  this 
work,  a  MAPOD  demonstration  study  is  presented.  In  addition,  the  benefit  of  the 
automated  data  analysis  algorithms  in  the  MAPOD  evaluation  process  is  highlighted. 

CIRCUMFERENTIAL  FEATURE  EXTRACTION  FOR  CRACK  DETECTION 

Simulated  studies  were  first  used  in  the  demonstration  of  circumferential  feature 
extraction  methods  for  crack  detection  around  fastener  sites.  Figure  1(a)  presents  a  plot  of 
the  reactance  component  of  impedance  with  respect  to  angular  location  in  the 
circumferential  direction  with  the  radial  position  set  to  9  mm.  Two  levels  of  probe  tilt  (0° 
and  2°,)  and  three  crack  sizes  (of  0  mm,  1  mm  and  2  mm  in  length)  are  presented.  Of 
particular  interest  is  a  significant  localized  peak  whose  magnitude  is  related  to  crack  length. 
There  is  also  a  second  peak  due  to  an  increased  probe  tilt  in  the  reactance  data;  however,  it 
is  much  broader  with  much  similarity  to  a  cosine  function  in  profile  and  period.  Although 
not  presented,  very  similar  trends  were  also  observed  for  the  cases  of  an  (a)  asymmetric 
gap  between  fastener  and  hole  (with  an  aluminum  fastener)  and  (b)  linear  liftoff  variation 
(with  a  titanium  fastener)  [6], 

To  extract  a  measure  of  this  localized  crack  feature  found  in  the  circumferential 
direction,  an  approach  is  proposed  through  the  estimation  of  the  corresponding  parameter 
in  a  model  characteristic  function.  The  feature  vector  in  the  circumferential  direction  can 
be  represented  by  four  model  components,  (1)  a  sinusoidal  function  representing  an 
asymmetric  response  of  a  typical  non-flaw  condition,  (2)  a  Gaussian  function  representing 
a  localized  response  associated  with  a  radial  crack,  (3)  constant  offset  value  related  to  the 
overall  axisymmetric  response  due  to  probe,  sample  geometry  and  material  properties,  and 
(4)  random  measurement  noise.  A  general  form  of  this  relation  is  given  by: 

f(9)=  J  +  X5pexp(-D>-pT)+Ccos(0+!#)+  X(9),  (1) 
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FIGURE  1.  Plots  of  (a)  reactance  component  along  circumferential  direction  at  r  =  9  mm,  for  varying  probe 
tilt  and  flaw  size  condition  (with  titanium  inserts),  (b)  Plot  of  all  circumferential  crack  measure  ( Bx )  results 
comparing  differences  in  flaw  size  and  insert  material  type  at  r  =  13  mm  [6]. 


where  Np  radial  fatigue  cracks  may  be  considered  with  varying  magnitude  and  phase  and  X, 
a  random  variable,  is  used  to  represent  measurement  noise.  Conveniently,  the  summation 
of  the  sinusoidal  components  can  be  reduced  to  a  single  cosine  function  of  magnitude  A 
and  phase  <f),  which  provides  a  straightforward  means  of  filtering  many  asymmetric  noise 
conditions.  To  evaluate  these  parameters,  a  nonlinear  least-square  estimation  approach  was 
used  to  minimize  the  error  between  the  characteristic  model  and  the  simulated  data. 

For  this  simulated  study,  the  model  was  limited  to  the  presence  of  a  single  crack  and  all 
phase  angles  were  set  equal  to  zero,  given  the  alignment  of  the  crack,  gap  asymmetry, 
probe  skew  and  probe  liftoff  asymmetry  with  the  x-axis.  Constraints  were  also  used  on  the 
parameter  D,  the  width  of  the  Gaussian  function,  to  limit  the  circumferential  extent  of  the 
peak  response.  Figure  1(b)  presents  a  comparison  of  all  of  the  circumferential  crack 
measure  ( Bx )  samples  comparing  differences  in  crack  size  and  insert  material  type  for  the 
probe  location  of  r  =  13  mm.  Of  particular  interest  is  that  the  vast  majority  of  variation  is 
due  to  the  insert  material  type  with  very  little  variability  due  to  the  other  noise  factors. 
Since  the  insert  material  can  be  found  by  eddy  current  measurements  made  at  the  center  of 
the  fastener,  this  trend  provides  a  feature  for  crack  sizing  with  excellent  noise  invariance. 

To  validate  the  proposed  feature  extraction  methodology,  a  set  of  experimental  data  for 
hidden  cracks  around  fasteners  in  multilayer  structures  was  analyzed.  Details  concerning 
the  experimental  design  and  test  specimens  are  presented  in  prior  work  [6].  An  automated 
signal  classification  algorithm  was  developed  based  on  the  proposed  circumferential  crack 
feature  extraction  algorithm  with  sensitivity  to  the  localize  Gaussian  response  above  and 
below  the  hole  associated  with  the  presence  of  a  crack  while  rejecting  any  step  changes  in 
the  response.  The  analysis  routine  was  applied  to  a  total  of  132  no  crack  fastener  sites  and 
39  crack  cases  with  lengths  ranging  from  0.027"  to  0.169".  When  flaws  greater  than  0.067" 
are  considered  and  a  simple  threshold  classifier  is  applied  to  the  circumferential  measure, 
all  flaws  were  detected  and  a  false  call  rate  less  than  1%  was  achieved. 


MODEL-BASED  FASTENER  SITE  AND  PART  EDGE  REMOVAL  ALGORITHM 

Although  the  proposed  circumferential  feature  extraction  methodology  is  beneficial  for 
distinguishing  crack  features  in  the  presence  of  certain  asymmetric  hole  features,  liftoff  and 
probe  tilt,  other  complex  features  of  aircraft  structures  can  hinder  the  direct  application  of 
the  approach.  For  the  experimental  demonstration,  all  cracks  were  located  perpendicular  to 
the  line  of  fasteners  and  thus  do  not  require  a  special  means  to  distinguish  them  from 
adjacent  fastener  sites.  For  general  inspections,  the  presence  of  adjacent  fastener  sites  and 
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part  edges  in  close  proximity  can  complicate  the  search  for  cracks  located  360°  around  the 
hole  of  interest.  In  addition,  they  can  also  contribute  to  the  eddy  current  response  in  crack 
regions  and  thus  impact  the  estimated  magnitude  degrading  sizing  accuracy.  There  is  a 
clear  need  for  model-based  feature  extraction  schemes  to  also  compensate  for  adjacent 
fastener  sites  and  part  edges. 

To  both  accurately  and  efficiently  represent  fastener  sites  and  part  edges  in  eddy  current 
measurement  models,  estimation  using  approximate  representations  was  investigated.  A 
collection  of  voltage  measurements  Vy  are  considered  at  [x„yi\  where  i  =  1,2,... TV,  data 
samples  for  the  two  measurement  components  (/'  =  1 ,2)  representing  the  real  (in-phase)  and 
imaginary  (quadrature)  parts.  The  measurement  model  can  be  approximated  by 
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where  RkJ  represents  the  response  due  to  an  unflawed  specimen  (without  fastener  sites),  Sy 
represents  the  response  for  each  fastener  site  k  =  1,2 ,..JV*  including  all  cracks  and 
asymmetric  hole  features,  and  Ey  represents  noise  due  to  measurement  error  (ei:j)  and 
coherent  noise  due  to  systematic  model  error  (Sy). 

The  fastener  site  measurement  model  is  first  considered.  The  proposed  model 
including  a  single  fatigue  crack  can  be  decomposed  into  axisymmetric  (ghk)  and  higher 
order  components: 


sij,k  =  8 j,k  ( Ek )  +  Bj,k  ( Ek )  exp(-  Dj,k  (< %,k  ~  (Pj,k } )+  cj,k  ( Ek )  cosfe,  +  (/>J  k )+  X, 


>J,k 


(3) 


where  each  fastener  site  is  centered  at  [xo,k,yo.k\,  and  rik  =  ^(x;  -  x()k  )2  +  (x-Tox)2  » 
6i  k  =  tan-1  [(y .  -  y()  k  )/(x.  -  x0  k )] .  To  efficiently  and  accurately  represent  the  axisymmetric 

response  at  a  general  fastener  site  as  shown  in  Figure  2,  a  two  part  function  is  used  to 
decompose  the  measurement  data 
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A  polynomial  representation  was  found  to  ideally  represent  the  primary  response  and  subtle 
differences  in  fastener  sites  using  few  parameters  and  minimal  function  evaluation  time. 
The  number  of  polynomial  terms,  TV/,  of  10  was  found  to  produce  good  results.  To  ensure 
the  response  converges  to  zero  as  becomes  large,  an  exponential  decay  term  was  added. 
Continuity  of  the  expression  for  all  r,,/c  is  maintained  through  the  fitting  of  the  exponential 
decay  term  (in  the  neighborhood  of  /y)  using  the  results  of  the  polynomial  fit,  Ajxi-  For  the 
steel  fasteners  present  with  titanium  fasteners,  the  saturated  signal  found  at  the  center  of 
fastener  response  can  be  addressed  using  an  additional  region  represented  by  a  polynomial. 

For  the  first  order  components  of  the  fastener  fit  problem,  the  solution  vector  fastener 
center  and  axisymmetric  response  is  given  by  z  =  [xoy,  yo,k,  A/kj\,  for  j  measurement 
components,  k  fastener  sites  and  /  polynomial  terms.  Additional  unknowns  for  the  part 
(edge),  crack  and  noise  terms  must  also  be  included  to  achieve  good  results  for  the  global 
error  minimization  problem.  Since  the  global  solution  for  all  of  these  unknowns  cannot  be 
readily  solved  as  a  large  single  error  minimization  problem  quickly  and  reliably,  the  new 
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FIGURE  2.  Plots  of  experimental  axisymmetric  fastener  site  responses  in  (a)  linear  and  (b)  log  scales. 


solution  strategy  presented  here  focuses  on  three  steps:  1)  a  heuristic  approach  using  a 
physical  understanding  of  the  sources  of  greatest  error,  2)  a  least-square  estimation 
approach  to  solving  for  the  polynomial  response  quickly  and  accurately,  and  3)  an  iterative 
approach  to  improve  model  solutions  for  overlapping  fastener  site  and  part  edge  regions. 

The  parameter  estimation  approach  was  designed  to  simplify  the  global  estimation 
problem  into  a  series  of  localized  estimation  problems  combined  with  iterative  schemes  to 
address  coupling  between  the  parameters  across  all  of  the  data.  The  first  task  is  to  estimate 
the  location  of  the  fastener  site  centers.  Figure  3(a)  displays  an  image  plot  of  the  original 
measurement  data  for  the  in-phase  component  ( Vx )  containing  10  fastener  sites  (9  of 
titanium,  1  of  steel).  To  achieve  precise  hole  removal,  evaluation  of  the  fastener  site  center 
is  performed  at  a  resolution  smaller  than  the  scan  step  size.  Correlation  methods  using 
iterations  of  varying  resolution  in  2D  were  used  to  calculate  the  initial  hole  centers. 
Fastener  site  material  type  is  then  evaluated  based  on  the  measurement  response  at  each 
hole  center.  Both  the  fastener  site  and  crack  characteristic  response  will  vary  with  fastener 
material  type.  Next,  data  in  regions  far  from  the  hole  center  but  near  the  panel  edge  is  used 
to  estimate  the  characteristic  response  due  to  the  panel  edge.  Figure  3(b)  displays  an  image 
plot  of  the  processed  data  with  hole  feature  extraction,  presenting  the  characteristic 
response  produced  as  the  probe  approaches  the  panel  edges  in  the  top  and  bottom. 
Expressions  for  the  measurement  response  model  at  the  panel  edges  in  the  y-direction  are 
given  by 

Rij  =  «o j  +  Poj  exp[-  A j Cc  - Pi,j  ~ P:,jxi  f  ]+  Yoj  exp[-  Yx j (}’,  ~ r2J  ~ Y\jxi  f\-  (5) 

The  tail  of  a  Gaussian  function  is  used  to  represent  the  decay  of  the  edge  response  as  the 
probe  is  moved  toward  the  center  of  the  part.  Also,  cross  terms  between  the  x  and  y 
directions  are  included  to  address  varying  alignment  between  part  edges  and  scan 
directions. 

Next,  estimation  of  the  axisymmetric  fastener  site  model  parameters  (Aj^i)  are 
evaluated  in  conjunction  with  an  heuristic  iterative  approach.  Local  regions  are  defined 
around  each  fastener  site  center  to  estimate  the  axisymmetric  model  parameters.  The  order 
of  the  fastener  site  evaluations  is  dependent  upon  fastener  material  type  and  location  in  the 
image  data.  Steel  fasteners  produce  significantly  larger  eddy  current  responses  with  respect 
to  titanium  and  aluminum  fasteners  and  typically  result  in  greater  error  between  the  model 
and  actual  data.  Thus,  the  iterative  process  begins  with  these  fastener  sites  for  model 
estimation.  Adjacent  fasteners  to  steel  fasteners,  non-end  fasteners  and  end  fasteners  are 
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FIGURE  3.  Image  plots  of  (a)  original  measurement  data  for  in-phase  component  ( Vx)  and  corresponding 
(b)  processed  data  with  hole  feature  extraction  and  (c)  processed  data  with  hole  and  edge  feature  extraction. 


then  subsequently  addressed  according  to  differing  levels  of  potential  model  error.  The 
least  square  error  estimation  is  thus  performed  in  an  iterative  sense  for  both  the  polynomial 
model  representation  (Ajxi)  and  also  the  fastener  center  [xo,k,yo,k]  and  their  convergence  is 
evaluated.  Figure  3(c)  displays  an  image  plot  of  the  processed  data  with  both  hole  and  part 
edge  feature  extraction.  For  this  specimen  with  three  cracks  located  around  fastener  sites, 
the  crack  features  are  more  clearly  observed.  In  addition,  other  weak  features  associated 
with  the  surface  condition  (producing  localized  changes  in  liftoff  due  to  bumps  or  missing 
paint)  were  also  observed.  A  complete  automated  process  performs  this  feature  extraction 
algorithm  in  approximate  60  sec  for  a  10  hole  panel,  providing  far  greater  accuracy  and  a 
10X  improvement  in  speed  over  prior  experience  with  direct  global  estimation  methods. 

Two  additional  refinements  to  the  estimation  problem  are  proposed  to  address  error  in 
the  axisymmetric  fastener  site  model  between  two  adjacent  fasteners  and  the  presence  of 
cracks  in  the  measurement  data.  First,  the  approximate  model  proposed  for  representation 
of  fastener  sites  assumes  that  there  is  no  significant  interaction  between  the  eddy  currents 
generated  between  adjacent  fasteners  and  thus  no  error  in  the  superposition  of  adjacent 
fastener  site  models.  To  validate  the  accuracy  of  this  assumption  for  the  problem, 
simulated  studies  in  VIC-3D®  were  performed.  Numerical  simulations  were  performed  for 
a  single  fastener  site  and  two  fastener  sites  separated  by  0.75"  (19  mm)  as  shown  in  Figure 
4(a).  The  single  fastener  site  result  was  applied  to  both  fastener  site  locations  and  the  data 
was  superimposed  to  compare  results  with  the  exact  numerical  case  for  the  two  adjacent 
fastener.  Figure  4(b)  shows  an  image  plot  of  the  percent  error  between  the  superposition  of 
axisymmetric  fastener  site  models  with  respect  to  full  numerical  solution.  The  peak  error 
of  2.1%  can  be  found  at  a  point  equidistant  between  the  two  fastener  centers.  This  adjacent 
fastener  superposition  error  corresponds  well  with  the  remaining  error  observed  between 
fastener  sites  in  the  experimental  data  shown  in  Figure  3(c).  Initially,  error  weighing  in  this 
region  is  used  to  improve  the  estimation  of  the  fastener  site  center  and  the  polynomial 
representation.  Inclusion  and  estimation  of  the  adjacent  fastener  superposition  error  in  the 
model  is  recommended  for  future  work.  Large  crack  signals  can  also  impact  both  proper 
centering  and  fastener  model  parameter  estimation.  Upon  detection  of  the  crack  condition, 
error  weighing  in  the  neighborhood  of  the  crack  was  also  used  in  this  study.  For  future 
work,  the  inclusion  of  a  crack  model  in  the  estimation  problem  is  recommended. 
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FIGURE  4.  Image  plots  of  (a)  simulated  results  for  two  adjacent  titanium  fastener  sites  and  (b)  percent  error 
between  between  superposition  of  axisymmetric  fastener  site  models  with  respect  to  full  numerical  solution. 


DISCUSSION  OF  MODEL-ASSISTED  POD  EVALUATION  AND  AUTOMATED 
DATA  ANALYSIS 


A  demonstration  is  also  presented  for  a  full  model-assisted  (FMA)  POD  methodology 
incorporating  computer  simulation  for  the  inspection  of  cracks  around  fastener  sites  in  a 
two  layer  aircraft  wing- type  structure  inspection  performed  with  an  eddy  current  technique. 
To  validate  the  proposed  model-assisted  POD  methodology,  a  set  of  experimental  data  for 
hidden  cracks  around  fasteners  in  multilayer  structures  was  analyzed  [6],  Using  methods 
and  experimental  data  presented  in  the  prior  sections,  the  new  feature  extraction  algorithms 
and  automated  signal  classification  routines  were  successfully  applied  to  the  experimental 
data  set  demonstrating  the  ability  to  detect  small  cracks  around  fasteners  while  maintaining 
a  low  false  call  rate.  VIC-3D®,  a  commercial  software  package  based  on  the  volume 
integral  method  [1],  was  used  to  perform  the  simulations  and  found  to  address  a  great 
majority  of  the  model  variables.  Simulated  parametric  studies  were  performed  for  varying 
crack  size  and  type,  considering  both  through  and  comer  crack  geometries.  The  simulated 
and  experimental  data  for  a  given  probe  and  inspection  geometry  were  equated  using  a 
calibration  procedure  developed  in  prior  work  [6].  Figure  5(a)  presents  a  comparison  of  the 
experimental  and  simulated  data  for  varying  crack  length  and  crack  type.  In  general,  there 
is  good  agreement  between  the  two  data  sets  with  some  error  for  select  experimental  data 
points.  A  Monte  Carlo  simulation  was  performed  using  the  noise  distribution  for  the 
unflawed  fastener  sites,  the  comer  crack  model,  and  the  through  crack  model,  to  populate 
the  full-model  assisted  data  sets  for  POD  evaluation.  Additional  details  on  the  FMA 
methodology  applied  to  this  problem  can  be  found  in  [7]. 

Figure  5(b)  presents  a  comparison  of  the  POD  results  for  the  experimental  and  full- 
model  assisted  approaches.  A  good  match  was  achieved  between  experimental  and  full- 
model  assisted  (FMA)  approaches  where  the  FMA  POD  curve  was  found  to  be  within  the 
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FIGURE  5.  (a)  Comparison  of  experimental  and  simulated  data  for  varying  crack  length  and  type,  and  (b) 
corresponding  POD  evaluation  results  for  experimental  and  full  model-assisted  POD  studies. 
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confidence  bounds  (95%)  of  the  empirical  POD  curve.  From  this  study,  automation  was 
found  to  play  a  key  role  in  the  success  of  the  model-assisted  evaluation  of  the  NDE 
technique.  First,  critical  NDE  system  parameters  can  better  controlled  using  automated 
scanning  systems.  In  particular,  there  is  less  opportunity  for  variation  related  to  human 
operation  of  equipment,  which  can  be  significant  given  the  variation  in  skill  and  experience 
in  inspectors.  Second,  the  proper  application  of  automated  signal  classification  algorithms 
has  the  potential  to  also  address  sensitivity  to  certain  NDE  measurement  and  part  factor 
variation.  In  particular,  algorithms  processing  noise  invariant  features  in  the  eddy  current 
data  as  observed  in  the  circumferential  direction  can  be  used  to  minimize  sensitivity  to 
asymmetric  hole  response  features  and  lift-off  for  the  fastener  crack  problem.  Applying 
greater  control  and  decreasing  sensitivity  to  NDE  system  parameters  results  in  greater 
success  of  MAPOD  methods  to  accurately  model  the  NDE  measurement  signals. 

CONCLUSIONS  AND  RECOMMENDATIONS 

New  model-based  feature  extraction  methods  were  presented  to  improve  sensitivity  to 
crack  features  in  data  with  coherent  noise  signals  while  also  improving  processing  time. 
First,  data  analysis  methods  in  the  circumferential  direction  around  the  fastener  hole  were 
developed  to  better  characterize  fatigue  cracks  with  improved  noise  invariance.  Second,  a 
novel  feature  extraction  method  fitting  approximate  models  to  data  associated  with 
geometric  part  features  such  as  adjacent  fastener  sites  and  panel  edges  was  developed  and 
demonstrated.  Model-assisted  probability  of  detection  results  are  also  presented 
highlighting  the  benefit  of  the  automated  signal  classification  algorithms.  Good  agreement 
was  found  between  the  experimental  and  full-model  assisted  POD  evaluation  results. 
Future  work  is  planned  to  further  refine  the  algorithms  for  improve  crack  characterization 
and  advanced  probe  designs  incorporating  GMR  sensors.  Significant  challenges  also 
remain  in  developing  model-assisted  POD  procedures,  including  improvements  to  the 
existing  computer  modeling  software,  optimization  of  method  to  combine  both  test  samples 
and  computer  simulations,  and  determination  of  the  uncertainty  in  the  evaluation  process. 
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